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FOREWORD 


The  Maintenance  and  Logistics  Training  Research  Technical  Area  of  the  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  conducts  re¬ 
search  into  the  influence  of  training  techniques,  devices,  and  delivery  systems 
on  individual  soldier  job  performance.  In  support  of  this  mission,  the  Transi¬ 
tional  Performance  Aid  (TPA),  a  job  aid,  was  examined  in  terms  of  user  accep¬ 
tance  and  performance  enhancement  before  its  introduction  into  the  classroom. 

This  research  was  conducted  in  the  context  of  the  Training  Technology 
Field  Activity  (TTFA)  at  the  Quartermaster  School  in  Fort  Lee,  Virginia.  This 
TTFA  site  and  others  located  at  Fort  Knox,  Kentucky,  Fort  Rucker,  Alabama,  and 
Gowen  Field,  Idaho  serve  as  test  beds  for  application  of  the  latest  in  train¬ 
ing  technology  and  for  research  to  identify  promising  new  training  methods. 

The  results  of  the  research  reported  here  indicate  that  students  can  easily 
apply  the  TPA  in  the  classroom  and  would  embrace  its  use  as  a  job  aid.  On  the 
basis  of  this  research,  the  TPA  has  been  implemented  in  the  classroom  with  the 
intention  of  training  the  students  in  its  application  as  a  job  aid. 


EDGAR  M.  JOHNSON 
Technical  Director 
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76C  TRANSITIONAL  PERFORMANCE  AID:  AN  EXAMINATION  OF  CLASSROOM  USE 


EXECUTIVE  SUMMARY 


Requirement : 

Determine  the  adequacy  of  the  Transitional  Performance  Aid  (TPA)  jointly 
developed  to  aid  the  soldier  in  the  field  by  three  U.S.  Army  organizations: 
the  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI),  the 
Quartermaster  School  (QMS)/  and  the  Training  and  Doctrine  Command  Training 
Technology  Agency  (TTA) -  Areas  of  evaluation  include  the  format/  the  suffi¬ 
ciency  of  the  accompanying  instructional  materials/  and  the  reaction  of  the 
users  following  a  preliminary  classroom  introduction. 


Procedure : 

The  research  was  conducted  in  two  phases;  in  each  phase/  students  of  the 
Equipment  Records  and  Parts  Specialist/  Advance  Individual  Training  Course/ 
76C10  served  as  participants. 

In  Phase  I,  10  students  received  brief  instructions  on  the  structure, 
format,  and  application  of  the  TPA.  Following  this  instruction,  they  used 
the  TPA  to  complete  a  test  on  technical  subject  matter  for  which  they  had  re¬ 
ceived  no  training. 

In  Phase  II,  four  classes  (totaling  122  students)  participated.  Two 
classes  of  students  were  instructed  on  the  use  of  the  TPA.  The  TPA  was  an 
available  resource  to  be  used  by  the  student  on  a  voluntary  basis  for  one  seg¬ 
ment  (11  days)  of  the  course.  The  performance  of  these  classes  was  compared 
to  that  of  two  standard  classes  (No  TPA)  conducted  concurrently  with  the  TPA 
classes. 


Findings : 

Phase  I  revealed  that  students  understood  the  TPA's  format  and  accurately 
applied  it  and  that  the  TPA  instructional  materials  were  adequate.  In  addi¬ 
tion,  Phase  II  indicated  that  students  voluntarily  applied  the  TPA,  wanted  to 
use  it  throughout  the  course,  and  wanted  to  use  it  in  the  field.  Additionally, 
TPA  usage  in  the  classroom  does  not  adversely  affect  test  performance  and  can 
reasonably  be  associated  with  improved  performance. 


Utilization  of  Findings: 

On  the  basis  of  these  findings,  the  school  has  initiated  the  implementa¬ 
tion  of  the  TPA  in  the  classroom.  The  principal  intent  of  this  classroom  im¬ 
plementation  is  to  have  the  students  become  familiar  with  and  accustomed  to 
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using  the  TPA  so  that  it  will  be  recognized  as  a  resource  to  be  applied  in 
the  field. 
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76C  TRANSITIONAL  PERFORMANCE  AID:  AN  EXAMINATION  OF 

CLASSROOM  USE 


BACKGROUND 

The  U.S.  Army  Research  Institute  (ARI),  in  conjunction  with  the 
TRADOC  Training  Technology  Agency  is  engaged  in  research  and  development 
to  increase  Army  training  effectiveness  through  the  implementation  of 
improved  instructional  methods  and,  where  appropriate,  advanced 
technologies.  The  four  Army  Training  Technology  Field  Activities  (TTFA) 
currently  participating  in  this  program  are:  Quartermaster  School  (QMS), 
Fu ;  »;  Lee,  VA;  Armor  School,  Fort  Knox,  KY;  Aviation  School,  Fort  Rucker, 
AL;  and  Gowen  Field,  ID.  The  Transitional  Performance  Aid  (TPA) ,  a  job 
aid,  examined  here  was  developed  under  the  Fort  Lee  TTFA  Program  for  the 
Equipment  Records  and  Parts  Specialist,  76C  Military  Occupational 
Specialty  (MOS) .  From  QMS,  education  specialists  from  the  Directorate  of 
Training  and  Doctrine,  and  subject  matter  experts  from  the  Enlisted  Supply 
Department  worked  together  with  research  psychologists  from  ARI  to  create 
the  TPA.  The  scope,  content  and  format  was  a  result  of  this  joint  effort. 

The  necessity  for  the  TPA  is  a  function  of  the  variety  of  duty 
positions  and  the  working  conditions  at  the  duty  site.  In  training  the 
76C,  a  principal  difficulty  is  the  integration  of  the  large  number  of 
distinct  actions  which  must  be  performed  for  each  of  the  four  duty 
positions.  Moreover,  once  at  the  duty  site,  the  76C  often  has  no  technical 
supervisor  to  ask  for  advice  while  operating  in  a  key  position  in  a 
complex,  highly  procedural i zed  supply  system.  The  TPA  integrates  and 
^ganizes  information  from  the  official  publications  which,  although 
useful  and  necessary  as  references,  do  not  provide  an  integrated  picture 
of  the  decisions  and  actions  required  to  execute  the  jobs.  The  TPA 
provides  this  integration  by  depicting  an  overall  view  of  the  full  set  of 
required  procedures  and  doctrinal  references  in  a  single  document.  It 
consists  of  flowcharts  keyed  to  a  reference  matrix.  The  flowchart  shows 
the  sequence  of  actions  to  be  performed,  while  the  reference  matrix 
directs  the  user  to  the  Army  publication  which  states  exactly  how  the 
action  is  performed.  Figure  1.  illustrates  these  features  of  the  two 
sections.  A  more  complete  description  of  the  TPA  and  its  development  is 
presented  in  Dressel,  Tremont,  and  Kessler  (1986). 

It  was  intended  that  the  TPA  would  be  introduced  into  the  program  of 
instruction  (POI)  primarily  to  allow  the  students  the  opportunity  to 
become  familiar  with  it  and  to  practice  using  it  in  a  variety  of 
situations.  Training  the  students  to  use  the  job  aid  while  in  school  would 
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increase  the  probability  of  its  use  and  the  utility  of  the  TPA  in  the 
field.  In  this  sense  the  TPA  is  comparable  to  other  job  resources  such  as 
technical  manuals  (TM) ,  Updates  and  microfiche;  to  be  used  in  the  field, 
it  must  be  taught  in  the  school. 

In  concept,  students  would  receive  the  TPA  early  in  the  9  week  4  day 
course  and  have  it  as  a  resource  to  use  throughout  the  course.  The 
soldiers  would  then  take  the  TPA  to  their  initial  duty  assignment  upon 
graduation.  There,  they  would  use  it  as  a  job  aid  until  mastering  their 
respective  jobs. 

However,  before  a  decision  could  be  made  to  incorporate  the  TPA  into 
the  classroom  and  send  it  with  the  students  to  the  field,  it  was  necessary 
to: 

1)  develop  and  evaluate  training  materials  concerning  the 
format  and  application  of  the  TPA,  and 

2)  determine  students'  response  to  and  use  of  the  TPA 
after  familiarization  conducted  during  a  typical  segment  of 
the  course. 

These  activities  were  conducted  in  Phases  I  and  II  respectively  of  the 
current  research. 

PHASE  I  APPLICABILITY  DEMONSTRATION 

ARI  researchers  trained  and  tested  a  group  of  ten  76C10  students  to 
determine  if  they  could  use  the  TPA.  The  randomly  selected  students  had 
been  in  the  course  from  2-6  weeks.  They  received  a  single  45  minute 
session  of  platform  instruction  consisting  of  overhead  transparencies, 
lecture,  and  question  probes.  This  instructional  approach  is  very  similar 
to  the  procedure  followed  in  the  standard  classroom.  The  instruction 
focused  on  the  format  of  the  TPA,  the  interpretation  of  flowcharts  and  how 
to  use  the  TPA.  Following  this  instruction,  which  included  applied 
problems,  the  students  received  a  15  item  multiple  choice  test  which  is 
found  in  Appendix  A.  Successful  completion  of  this  test  required: 
understanding  of  the  TPA  structure  in  order  to  answer  questions  specific 
to  the  TPA;  and  knowledge  of  TPA  application,  sufficient  to  answer 
questions  concerning  technical  subject  matter  to  which  the  students  had 
not  been  exposed  in  either  the  training  session  or  the  course. 

The  students  required  approximately  one  hour  to  complete  the  test. 

The  test  required  the  students  to  use  the  TPA  in  conjunction  with  the 
Maintenance  Management  Update  to  answer  questions  regarding  the 
performance  of  procedures  from  The  Army  Maintenance  Management  System 
(TAMMS) . 


TAMMS  is  the  system  which  directs  the  preparation  and  management  of  forms 
and  records  required  to  sustain  maintenance  activities.  TAMMS  was 
selected  since  its  subject  matter  represents  a  series  of  activities  which 
are  not  dependent  upon,  and  therefore  not  likely  to  be  influenced  by, 
prior  course  instruction  concerning  the  other  interdependent  duty 
positions  within  the  MOS.  The  Maintenance  Management  Update  is  a  document 
published  semi-annually  which  is  a  current  compilation  of  those  technical 
manuals  and  regulations  which  direct  the  actions  of  TAMMS  and  indicate  any 
changes  in  procedure  since  the  previous  publication. 

The  test  results  indicated  that  the  students  could  answer  questions 
specific  to  the  TPA  and  questions  concerning  TAMMS  actions.  Students 
responded  correctly  to  92%  of  the  test  items.  The  majority  of  the 
students  (70%)  committed  one  error  or  less.  Students  also  reported  having 
no  trouble  interpreting  the  flowcharts  or  finding  the  task  in  the 
reference  section  of  the  TPA.  All  students  responded  favorably  when 
queried  informally  by  the  researchers  on  the  possible  use  of  the  TPA  in 
the  course  and  in  the  field. 

PHASE  II  CLASSROOM  FAMILIARIZATION 

Following  the  applicability  demonstration,  it  was  necessary  to 
determine: 

a)  the  extent  to  which  students  would  use  the  TPA  in  the  standard 
classroom  situation; 

b)  student  reaction  to  the  TPA;  and 

c)  the  influence  of  TPA  usage  on  classroom  performance  measures. 

It  should  be  remembered  that  the  TPA  was  designed  as  a  job  aid.  Its 
introduction  into  the  classroom  was  necessary  in  order  that  the  students 
could  become  familiar  with  it  and  become  proficient  in  its  use  prior  to 
the  time  they  graduate  and  take  it  with  them  to  the  job.  Demonstrated  or 
reported  use  in  the  classroom  environment  and  the  lack  of  impaired  class¬ 
room  performance  as  a  result  of  TPA  use,  in  conjunction  with  the  positive 
results  Phase  I  concerning  ease  of  use,  would  provide  support  for  a  deci¬ 
sion  to  make  the  TPA  a  standard  component  of  the  course.  Evidence  that 
the  use  of  the  TPA  improves  performance  in  the  classroom,  perhaps  by  act¬ 
ing  as  an  advance  organizer,  would  provide  an  additional  reason  for  using 
the  TPA.  However  the  failure  to  find  such  evidence  would  not  argue 
against  its  use;  instructors  were  given  no  guidance  to  actively  use  the 
TPA  either  as  an  advance  organizer  or  in  any  way  depart  from  the  standard 
program  of  instruction. 
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PROCEDURE 

To  make  these  determinations,  two  classes  of  students,  had  access  to 
the  TPA  during  a  typical  segment  (or  annex)  early  in  the  course.  As  a 
control,  standard  classes  were  conducted  concurrently  with  each  TPA  class. 
The  two  TPA  classes  totaled  60  students,  while  the  control  classes  totaled 
62  students.  Background  data  to  be  discussed  later,  was  gathered  for  all 
students.  The  annex  anployed  for  Phase  II  was  B  Annex  which  is  an  11  day 
(79  course  hours)  period  of  instruction  on  tasks  required  to  maintain  a 
prescribed  load  list  (PLL) .  A  PLL  represents  the  inventory  of  essential 
repair  parts  that  a  unit  needs  in  order  to  maintain  the  readiness  of  its 
equipment;  the  requisite  number  of  these  parts  should  either  be  on  hand  or 
on  order  at  all  times. 

The  usual  instructional  protocol  has  the  instructor  (one  of  three 
assigned  to  the  class)  lecturing  the  class,  frequently  using  an  overhead 
projector,  on  how  to  perform  a  given  task.  The  class  then  applies  and 
practices  the  task  by  completing  hand-out  problems  called  a  practical 
exercise (PE) .  The  standard  procedure  is  for  each  student  to  work 
individually  and  independently  to  complete  the  PE.  Upon  completion,  the 
student  takes  the  PE  to  the  instructor  who  quickly  inspects  it  and 
discusses  any  errors  with  the  student.  After  all  PEs  have  been  inspected, 
the  instructor  stands  before  the  class  and  clarifies  any  area  in  which  the 
students  had  difficulty.  This  cycle  is  repeated  for  each  block  of 
instruction  within  the  annex.  The  annex  concludes  with  a  written  test 
which  the  student  must  pass  (either  initially  or  upon  retesting)  in  order 
to  proceed  to  the  next  annex  of  instruction  where  the  lecture-PE  cycle  is 
continued.  This  protocol  was  not  modified  to  accommodate  the  use  of  the 
TPA.  The  TPA  was  simply  an  available  resource  for  students  in  the  TPA 
treatment  classes. 

ARI  researchers  provided  training  for  the  randomly  selected  classroom 
instructors  on  the  intent,  format  and  use  the  TPA  irtmediately  prior  to  the 
beginning  of  each  TPA  class.  This  training  was  very  similar  to  the 
training  given  the  students  in  Phase  I.  The  instructors  now  employed 
these  training  materials  to  instruct  their  students  on  the  TPA.  This 
instruction  required  approximately  one-half  hour.  Each  student  received  a 
copy  of  the  TPA  for  his  or  her  voluntary  use  during  the  annex.  Throughout 
the  annex,  the  students  had  the  opportunity  to  become  familiar  with  the 
TPA  and  employ  it  as  they  chose.  The  instructors  neither  forced  the  use 
of  the  TPA  nor  stressed  it  application.  The  classes  were  conducted  as 
usual. 

All  training  activities  of  the  76C  course  are  directed  to  prepare  the 
students  for  the  duties  and  responsibilities  which  await  them  in  the  unit. 
Here,  they  are  expected  to  follow  the  doctrine  and  procedures  prescribed 


by  numerous  technical  manuals  and  other  regulating  documents.  Throughout 
the  course,  the  students  are  trained  to  use  these  same  technical  materials 
for  all  classwork  including  examinations.  All  examinations  are  therefore 
"open  book  tests";  the  TPA  was  available  as  one  of  these  "books"  for  the 
TPA  treatment  classes. 

At  the  conclusion  of  B  Annex,  students  took  three  performance  tests. 

The  first  was  a  35  item  multiple  choice  test  (MCT)  developed  by  ARI  for 
QMS  as  part  of  the  TTFA.  This  test  systematically  examined  the  student's 
understanding  of  the  content  of  the  B  Annex  (see  Cormier,  1987,  for 
developmental  details) .  The  second  measure  was  the  standard  comprehensive 
practical  exercise  (CPE) .  This  is  a  series  of  problem  solving  exercises 
involving  PLL  actions.  The  final  measure  was  the  end  of  annex  test  (EOAT) 
which  while  similar  in  format  and  content  to  the  CPE,  was  used  to 
determine  whether  the  student  would  be  allowed  to  proceed  in  the  course. 

ARI  researchers  administered  the  60  minute  MCT  to  all  classes  and 
monitored  the  TPA  usage  during  it.  Following  the  MCT,  the  students  of  the 
TPA  classes  were  given  a  short  seven  item  questionnaire  concerning  their 
usage  and  ease  of  application  of  the  TPA.  The  questionnaire  is  shown  in 
Appendix  B.  Next,  all  classes  received  the  six  hour  CPE  which  the 
classroom  instructors  administered  and  ARI  monitored. 

This  monitoring  activity  during  the  MCT  and  CPE  was  simple  and 
straightforward.  The  researcher,  having  the  seating  plan  of  the  classroom 
and  the  instructor's  assurance  that  all  students  were  in  their  assigned 
seat,  annotated  the  seating  plan  during  the  MCT  and  the  CPE  once  the 
student  was  seen  referring  to  the  TPA.  No  attempt  was  made  to  measure  the 
relative  degree  of  TPA  application,  either  between  students  or  between 
technical  documents.  The  question  being  considered  was  whether  students 
recognized  the  TPA  as  a  source  of  information  and  demonstrated  this 
recognition  by  using  it  when  needed. 

The  following  day  the  instructors  reviewed  the  CPE  with  students. 

After  collecting  the  CPE,  QMS  test  officials  administered  the  EOAT  which 
the  school  allows  four  hours  to  complete.  ARI  researchers  were  not  onsite 
during  the  EOAT. 

RESULTS  AND  DISCUSSION 

Background  Data .  Background  data  were  collected  on  the  participating 
122  students  from  the  four  classes  and  analysed  to  determine  both  the  char¬ 
acteristics  of  these  students,  and  the  comparability  of  the  treatment  groups. 


The  data  included  the  students'  general  technical  (GT)  score  on  the  Armed 
Services  Vocational  Aptitude  Battery  ( ASVAB ) ,  clerical  (CL)  score  on  the 
ASVAB  and  service  component. 

The  GT  score  means  for  the  TPA  group  and  the  control  group  were 
109.03  (SD=10.40)  and  106.94  (SD=14.82)  respectively;  the  difference  was 
not  significant  (p>.05).  The  CL  scores  were  also  comparable  with  the  mean 
for  the  TPA  group  being  109.60  (SD=11.98),  while  the  control  group  was 
110.37  (SD=11.21) . 

A  chi  square  test  however,  indicated  the  two  groups  were  not 
comparable  (p<.05)  in  terms  of  service  component.  Overall  58%  of  the  TPA 
group  were  regular  army,  while  65%  of  the  control  group  were  reserves  or 
national  guard.  Closer  examination  of  these  data  indicated  that  the 
imbalance  was  created  by  the  second  control  class  which  was  93%  reserves/ 
national  guard.  When  this  class  was  deleted,  the  chi  square  test  indicated 
the  groups  to  be  comparable,  p>.05. 

This  creates  somewhat  of  a  dilemma.  While  intact  groups  were 
comparable  in  terms  of  ASVAB  components  (aptitude  measures)  they  were  not 
equivalent  in  terms  of  service  component  (possible  experiential  and 
motivational  differences).  Due  to  the  emphasis  of  this  effort  being  on 
the  user  reaction  to  the  TPA  rather  than  performance  gains,  the 
imprecision  of  the  treatment,  and  the  finding  that  the  discrepant  class 
was  in  the  control  group  (no  TPA  exposure  therefore  no  reaction),  it  was 
determined  to  include  the  data  from  all  four  classes.  The  performance  of 
this  second  control  class  will  be  noted  as  appropriate  in  the  analyses. 

Observations  and  Usage.  Most  students  (80%)  in  the  TPA  classes  were 
observed  using  the  TPA  during  the  MCT.  The  TPA  was  employed  less  often 
during  the  CPE  when  66%  of  the  students  referred  to  it. 

The  increased  application  of  the  TPA  for  the  MCT  may  be  due  to  the 
nature  of  the  test.  The  MCT  consists  of  discrete  items  whose  solutions 
are  largely  unrelated  to  those  of  adjacent  test  items.  This  influences 
how  the  students  search  for  item  solutions;  the  solution  for  adjacent  test 
items  may  be  pages  apart  in  the  same  technical  manual  or  in  different 
manuals.  This  creates  more  opportunities  to  refer  to  the  TPA.  On  the 
other  hand,  the  CPE  presents  a  series  of  task  scenarios  for  completion; 
here  once  the  student  locates  the  appropriate  section  of  the  TM,  many  of 
the  solutions  are  found,  thereby  reducing  the  number  of  information 
seeking  opportunities. 


It  should  also  be  recalled  that  the  TPA  does  not  offer  unique 
information,  i.e.  information  not  available  from  other  resource  documents. 
The  TPA  does  organize  the  sequence  of  task  events  and  directs  the  user  to 
the  specific  paragraphs  of  the  other  technical  documents  for  guidance. 
Therefore,  it  was  not  necessary  for  any  student  to  use  the  TPA  in  order  to 
perform  satisfactorily  on  the  tests.  However,  the  students'  perceived 
utility  of  the  TPA  application  is  evident  in  their  high  rate  of  TPA  usage. 

Questionnaire  data  collected  at  the  conclusion  of  the  MCT,  also 
indicated  the  students'  acceptance  of  the  TPA.  Generally,  the  students 
found  the  TPA  easy  to  understand  and  apply.  They  used  it  in  the  classroom 
in  various  ways  including  practical  exercises  (PEs)  and  examinations.  The 
overwhelming  majority  would  like  to  use  the  TPA  throughout  the  course 
(92%)  and  in  the  field  (90%) .  Additionally,  of  those  students  who  were 
observed  not  to  have  used  the  TPA  during  the  MCT,  75%  reported  accordingly 
on  the  questionnaire.  This  convergence  of  observed  and  reported  usage  can 
be  viewed  as  an  indication  of  the  veracity  or  accuracy  of  the  students' 
questionnaire  responses.  The  full  compilation  of  student  responses  and 
questionnaire  items  appears  as  Appendix  B. 

PERFORMANCE  DATA 

CPE 

It  was  originally  intended  that  the  CPE  data  be  compared  between  the 
two  treatment  groups.  However,  due  to  test  administration  and  scoring 
inconsistences,  these  data  were  unreliable  and  therefore  not  analysed.  As 
previously  noted,  the  CPE  is  a  series  of  job  scenario-based  problem 
solving  exercises  involving  PLL  actions.  The  CPE  consists  of  a  37  page 
test  booklet.  Six  hours  are  allowed  for  completion.  Normally,  when  the 
students  complete  the  CPE,  the  booklet  is  reviewed  by  the  instructors  to 
provide  feedback  to  the  students  rather  than  given  a  numerical  score.  For 
purposes  of  this  evaluation  only,  instructors  were  asked  to  score  each 
CPE.  After  scoring  differences  were  noted  in  the  first  pair  of  classes, 
the  school  staff  prepared  a  scoring  template  to  be  used  with  the  second 
pair  of  classes.  However  in  both  sets  of  these  classes,  not  all  of  the 
students  finished  the  CPE  in  class.  Since  the  instructors  viewed  it  as 
valuable  training  to  complete  the  CPE  and  also  considered  the  CPE  to  be  a 
good  study  aid  for  the  next  day's  EOAT,  students  were  allowed  to  complete 
the  CPE  in  the  barracks.  The  students  were  instructed  to  mark  the  place  in 
their  test  booklet  where  their  classroom  effort  ceased.  The  research 
intent  then  was  to  analyse  the  percent  correct  of  those  items  completed  in 
class.  However,  it  was  possible  for  students  while  in  the  barracks  to 
receive  help  and  alter  their  previously  completed  classwork.  Since  the  MCT 
and  EOAT  provided  objective  and  uniformly  scored  datum  sets,  no  purpose 
would  be  served  by  considering  the  CPE  data. 
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A  nested  analysis  of  variance  (ANOVA)  having  students  within  classes 
within  levels  of  treatment  (TPA  versus  standard)  was  performed  on  the 
percent  correct  of  those  items  attempted.  For  some  unknown  reasons, 
nearly  one-half  of  the  students  in  the  second  TPA  class  failed  to  complete 
the  final  items  of  the  MGT.  Rather  than  unduly  jeopardize  the  impact  of 
the  data  from  this  class,  this  less  stringent  dependent  measure  (as 
compared  to  percentage  correct  of  total  number  of  test  items)  was  applied 
to  all  four  classes.  Although  this  ANOVA  revealed  no  treatment  effect 
(F<1),  a  great  deal  of  variability  was  attributed  to  classes  within 
treatment.  To  examine  this  variability  between  classes,  a  one-way  ANOVA 
was  performed  on  the  data  from  the  four  classes.  This  analysis  indicated 
a  significant  (p<.05,F=5.67,df.=3,118)  difference  between  classes. 
Subsequent  tests  (Student-Newnan-Keuls  procedure)  and  non-pooled  contrasts 
showed  the  second  TPA  class  (the  nonccmpleting  class)  scored  lower  than 
the  three  remaining  classes  which  did  not  differ  from  one  another.  The 
mean  percentage  correct  for  each  class  is  shown  in  Table  1. 

The  nested/one-way  ANOVA  sequence  was  performed  again  selecting  only 
those  cases  in  which  the  students  had  been  observed  using  the  TPA.  The 
only  additional  finding,  from  this  series,  was  that  the  first  TPA  class 
performed  significantly  better  than  its  control  class. 

A  final  analysis  on  the  MOT  data,  using  only  TPA  classes,  was 
conducted.  This  analysis  considered  only  those  students  who  completed  the 
test.  The  performance  of  observed  TPA  users  was  compared  to  the 
performance  of  non-users.  A  simple  t-test,  adapted  to  accommodate  large 
differences  in  sample  size  (McGuigan,  1964)  indicated  that  TPA  users 
performed  significantly  better  (p<.01,df=52,t=3.04)  than  their  non-user 
classmates  (81.1  percent  correct  vs.  71.9  percent  correct,  respectively). 
Examination  of  the  aptitude  measures  show  that  users  and  non-users  were 
comparable  ( p> . 40 )  in  terms  of  GT  and  CL  scores  with  groups  means  of: 
110.12,  and  107.47  respectively  for  GT;  110.68  and  107.46  respectively  for 
CL.  While  TPA  users  were  not  inherently  "smarter"  than  their  non-user 
peers,  TPA  usage  is  associated  with  improved  MCT  performance. 

EOAT 

The  nested  ANOVA  was  performed  on  the  percentage  of  test  items 
correct  (all  students  in  all  classes  completed  this  test) .  The  ANOVA 
revealed  no  treatment  effects  but  considerable  variability  within  groups. 

A  one-way  ANOVA  indicated  significant  differences  between  classes 
(p<.05,F=9. ll,df=3, 118) .  Subsequent  tests  and  contrasts  indicated  a  mixed 


Table  1 


Mutiple  Choice  Test  (MCT):  Mean  Percentage  Correct  for  Each  Class 


FULL  CLASS 

TPA  USERS  Only 

CLASS 

N 

MEAN 

SD 

N 

MEAN 

SD 

TPA  1 

26 

82.27 

9.37 

22 

84.97 

5.56 

CTL  1 

24 

80.05 

9.69 

- 

- 

- 

TPA  2 

34 

74.92 

8.50 

24 

75.33 

9.66 

CTL  2 

38 

82.58 

7.39 

_ 

_ 

Table  2. 

End 

of  Annex  Test 

( EOAT ) : 

Mean  Percentage  Correct  for  Each  ( 

Full 

Class 

TPA 

Users  Only 

Class 

N 

Mean 

SD 

N 

Mean  SD 

TPA  1 

26 

91.50 

6.03 

22 

92.61  5.15 

CTL  1 

24 

91 . 96 

6.00 

- 

- 

TPA  2 

34 

95.70 

3.92 

24 

95.83  3.79 

CTL  2 

38 

96.70 

3.45 

_ 

_  _ 

|Q 


performance.  The  second  TPA  class  and  the  second  control  class  (mean 
scores  of  95.70  and  96.70  respectively),  each  performed  significantly 
better  than  the  first  TPA  and  control  class  pair  (mean  scores  of  91.50  and 
91.96  respectively).  The  performance  within  each  pair  of  classes  (TPA  vs 
control)  did  not  differ.  Consequently,  as  shown  in  the  nested  ANOVA,  the 
combined  performance  of  the  TPA  classes  did  not  differ  from  the  combined 
performance  of  the  control  classes.  The  mean  percentage  correct  for  each 
class  is  shown  in  Table  2. 


The  same  nested  ANOVA  was  performed  comparing  the  performance  of  the 
observed  users  of  the  TPA  (i.e.  those  students  who  had  used  the  TPA  during 
the  MCT)  to  performance  of  the  control  classes.  This  restriction,  to 
previously  observed  users,  did  not  alter  the  findings.  Again,  the  one-way 
ANOVA  was  applied  to  the  class  data  and  again  the  findings  did  not  differ 
from  those  found  for  the  full  class. 


Performance  Summars 


In  summary,  no  significant  treatment  effects  were  found  between  the 
TPA  and  control  conditions  when  using  the  EOAT  scores  as  the  dependent 
measure.  However,  significant  treatment  effects  were  found  when  using  the 
MCT  scores.  Here,  there  was  lower  performance  in  the  second  TPA  class 
(and  therefore  reduced  performance  for  the  TPA  condition  overall);  when 
considering  only  the  TPA  classes,  there  was  higher  performance  for  users 
of  the  TPA  than  for  their  non-user  classmates. 


It  is  not  suprising  that  the  MCT  should  be  more  sensitive  than  the 
EOAT  to  TPA  effects.  First,  the  mean  scores  on  the  EOAT  are  so  high,  with 
class  means  ranging  from  91.50  to  96.70,  that  ceiling  effects  reduce  the 
likelihood  of  demonstrating  differences  due  to  other  causes.  The  MCT  was 
clearly  more  difficult,  with  class  means  ranging  from  74.92  to  82.58. 
Second,  the  MCT  includes  job-related  material  that  may  not  have  been 
stressed  in  class.  The  EOAT  (and  the  CPE,  as  well)  includes  only  material 
which  has  been  stressed  to  the  students.  It  would  be  expected  that 
students  would  be  more  likely  to  use  the  TPA  during  the  MCT  than  during 
the  EOAT,  and  that  its  effects  would  be  more  readily  observable  on  MCT 
scores . 


While  this  cannot  be  checked  directly,  because  observations  of  TPA  use 
were  not  taken  during  the  EOAT,  it  should  be  noted  that  more  students  were 
observed  using  the  TPA  during  the  MCT  (80%)  ,  than  during  the  CPE  (66%) , 
which  is  highly  similar  to  the  EOAT. 


usage  could  include  its  use  as  a  review/surnmary  tool  before  annex  and  end 
of  course  testing.  The  instructors  could  also  enploy  the  TPA  when 
critiqueing  the  student's  work.  Although,  neither  of  these  applications 
would  greatly  alter  the  POI,  each  would  function  to  promote  classroom 
usage  and  concurrently  increase  the  visibility  of  the  TPA.  The  increased 
classroom  visibility  may  lead  the  student  to  place  a  higher  value  on  the 
TPA  as  a  job  resource.  Consequently,  the  student  would  exert  greater 
efforts  to  maintain  possession  of  this  valued  item  and  take  it  to  the 
field  where  it  is  applied  as  intended. 

A  field  evaluation  of  TPA  usage  by  the  76C  and  supervisor  reaction 
has  been  initiated. 
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APPENDIX  A 


TRANSITIONAL  PERFORMANCE  AID  APPLICATION  TEST 


Last,  First 


TRANSITIONAL  PERFORMANCE  AID 


TAMMS  TEST 


1.  What  is  the  last  action  you  take  when  a  dispatched  vehicle  is 
re  turned  . 

a.  Return  vehicle  to  assigned  parking  area. 

b.  Transfer  all  needed  information  to  a  new  DD  2970. 

c.  Receive  equipment  record  folder  from  operator. 

d.  Enter  the  time  of  return  on  the  DA  24C1. 

2.  Which  reference  figure  is  used  when  performing  the  TAMMS  task  entitle 
"Maintain  oil  analysis  records"? 


3.  Your  unit  gains  a  new  vehicle.  Which  people  receive  copies  of  the 
Gain  Report? 

a.  Those  specified  in  DA  PAM  738-730. 
t.  Those  specified  in  AR  710-2. 

c.  The  unit  commanders. 

d.  The  property  book  officers. 


What  does  a 


a .  Decision. 

b.  Task  action 
c  .  Connec  tor. 
d.  Stop. 


represent  on  a  flow  chart? 


mm®* 


SvSw 


.'VV 

5. 

Which  task  does  the  code  represent? 

a  . 

Maintain  PLL. 

b. 

Receive  repair  parts. 

c . 

Initiate  follow-up  action. 

d  . 

Initiate  cancellation  action. 

6. 

Which  form  is  used  to  request  DSL’  maintenance? 

a  . 

DA  2404. 

b. 

DD  314. 

c  . 

DA  2408-10. 

d  . 

DA  2407. 

7. 

As  a 

TAMMS  clerk,  what  do  you  do  with  the  dispatch  form  DA 

24  C 1  ? 

a  . 

Destroy  one  month  after  last  entry  in  column  1  has  been  closed 

out . 

b. 

Destroy  two  months  after  last  entry  in  column  1  has  been  closed 

out . 

c  . 

Destroy  three  months  after  last  entry  in  column  1  has 

been  closed 

ou  t . 

d  . 

Destroy  four  months  after  last  entry  in  column  1  has  beer,  closed 

out . 

8. 

You 

are  the  Ta-M.MS  clerk  about  to  dispatch  a  vehicle.  There 

are  no 

uncorrected  faults  listed  on  the  DA  2404.  What  action  do  you  now  take? 

a  . 

Send  the  PMCS  DA  2404  to  the  maintenance  supervisor. 

b  . 

Send  the  PMCS  DA  2404  for  dispatch  approval. 

c  . 

Prepare  DD  1970. 

d  . 

Dispatch  the  vehicle. 

9. 

When 

performing  task  which  references  are  used? 

a  . 

DA  PA.M  738-750  and  TM  38-L09-11. 

b . 

DA  PA.M  738-750  and  TB  43-0210. 

c  . 

T.M  38-L09-1 1  and  DA  PA.M  710-2-2, 

d  . 

T.M  38-L09-1 1  and  AR  18-10. 

10. 

What  symbol  would  be  entered  on  the  DD  314  when  scheduling 

a  rotation 

of  tire6 

2 

a  . 

T. 

b. 

E. 

c . 

H. 

d. 

R. 
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11.  As  a  TAMMS  clerk,  when  preparing  a  materiel  condition  status  report  . 
what  would  the  code  "A"  mean  in  block  3? 


a.  Army  Intelligence. 

b.  Active  Component. 

c.  Army  National  Guard. 

d.  Army  Reserve  units. 

12.  What  does  *'ERF“  mean  to  a  TAMMS  clerk? 

a.  Emergency  Record  File. 

b.  Equipment  Register  Format. 

c.  Equipment  Record  Folder. 

d.  Emergency  Response  File. 

13.  As  a  TAMMS  clerk  you  must  post  the  DA  2401  when  dispatching  equip¬ 
ment.  What  information  do  you  enter  in  column  h? 

a.  Type  of  equipment. 

b.  Requestor's  phone  number. 

c.  Expected  time  of  return. 

d.  Registration  number. 

14.  In  preparing  DA  2406  (monthly  MCSR),  which  form  do  you  need  to 
extract  information  from? 

a.  DD  314. 

b.  DD  2026. 

c.  DA  2408-20. 

d.  DA  2404. 

15.  DA  form  2408-5  is  used  in  which  TAMMS  task? 

a.  Prepare  a  materiel  condition  status  report. 

b.  Maintain  a  consolidated  equipment  log. 

c.  Maintain  oil  analysis  records. 

d.  Request  repair  or  modification  of  equipment. 


APPENDIX  B 


TRANSITIONAL  PERFORMANCE  AID  USER  SURVEY 


NAME: _ 

Last, 


First 


TRANSITIONAL  PERFORMANCE  AID  USER  SURVEY 


1.  How  often  did  you  use  the  TPA  during  this  PLL  Manual  Annex?  (Check 
only  one). 

21 _  Never 

13  Once  or  twice 

52  Sometimes  but  not  everyday 

8  Once  a  day 

i7  Several  times  a  day 

2.  Check  all  the  ways  you  used  the  TPA: 

911  PEs 

36  Studying 

63  Testing 

5  Did  not  use 

3.  How  easy  or  difficult  was  it  for  you  to  understand  the  flowcharts? 

411  Quite  easy  to  understand 

38  Fairly  easy  to  understand 

16  Borderline 

5  Fairly  difficult  to  understand 

2  Quite  difficult  to  understand 

4.  How  easy  or  difficult  was  it  for  you  to  find  the  task  reference  in  the 
back  of  the  TPA? 

471  Quite  easy  to  find 

31  Fairly  easy  to  find 

19  Borderline 

2  Fairly  difficult  to  find 

2  Quite  difficult  to  find 


18 


How  easy  or  difficult  was  it  for  you  to  locate  the  reference 
paragraphs  in  the  Update  after  reading  the  TPA? 


45Z  Quite  easy  to  locate 

38  Pairly  easy  to  locate 

14  Borderline 

3  Fairly  difficult  to  locate 

0  Quite  difficult  to  locate 

Uould  you  like  to  use  the  TPA  for  the  rest  of  the  course? 

92Z  Yes  8 _  No 

Uould  you  like  to  take  the  TPA  with  you  after  you  complete  the  course 

91X  Yes  9  No 


